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George  Sevmour,  Rav  Main,  Josephine  Randei, 
and  Barbara  Morris  are  with  the  Navy  Personnel 
Research  and  Development  Center,  San  Diego. 

A  16-dimension  survey  to  assess  study  skills 
ivas  designed  and  administered  to  705 
students  enrolled  in  five  Navy  schools.  After 
revision  based  on  reliability  analyses,  valid¬ 
ity  analyses  were  performed  at  two  of  the 
schools  by  correlating  survey  responses  with 
test  performance  measures.  At  one  school, 
correlations  indicated  that  those  students 
who  were  more  competitive  or  motivated, 
had  higher  concentration  or  memorization 
skills,  or  asked  more  questions  performed 
significantly  better  on  the  most  difficult  tests 
in  the  school.  At  the  second  school,  using 
partial  correlations  to  control  for  student 
ability,  study  factors  reliably  predicted  test 
performance,  irrespective  of  student  ability. 
Multiple  regression  coefficients  of  .618  and 
.379  supported  the  independent  contribution 
of  several  study  factors  to  test  performance. 
Selected  study  skill  training  resources  used 
by  the  military  are  identified  and  functional 
research  directions  are  described. 


The  opinions  expressed  in  this  research  report  are  those 
of  the  authors,  are  not  official,  and  do  not  necessarily  reflect 
the  views  of  the  U.  S  Navy  Department.  The  critical  com¬ 
ment  on  an  earlier  version  of  this  report  by  Jack  Edwards 
and  three  anonymous  reviewers  is  appreciated. 

Correspondence  concerning  this  article  should  be  ad¬ 
dressed  to  George  Edw  Seymour.  Instructional  Sciences 
Division:  Code  151.  Naw  Personnel  Research  and  Devel¬ 
opment  Center,  San  Diego,  California  92152-6800. 


□  In  traditional  academic  settings,  personal 
aids  or  skills  to  enhance  learning  usually  have 
been  limited  to  reading  comprehension,  mem¬ 
ory,  or  prose-learning  content  areas.  The  pri¬ 
mary  reason  is  that  both  teachers  and  theorists 
have  long  recognized  that  reading  and  its  com¬ 
prehension  are  fundamental  to  the  learning 
process  (Wittrock,  1974).  Thus,  prior  to  1940 
the  primary  topics  for  learning  how  to  study 
consisted  of  time  mangement,  systematic  note 
taking,  outlining  or  underlining,  and  selecting 
a  place  to  study  (Schulte  &  Weinstein,  1981). 

A  major  departure  from  this  limited  focus  of 
study  skills  occurred  with  the  formal  presenta¬ 
tion  and  then  later  acceptance  of  Robinson's 
(1946)  Survey-Question-Read-Redte-Review 
(SQ3R)  method.  (Robinson  referred  to  it  as  the 
"Survey  Q3R  Method.")  Although  the  SQ3R 
method  proved  extremely  useful  for  address¬ 
ing  specific  deficits  in  studying,  education  the¬ 
orists  recognized  that  the  composite  of  study 
skills  was  much  more  extensive  and  complex 
than  previously  thought.  In  particular,  the  in¬ 
vestigation  of  cognitive  issues,  as  opposed  to 
simple  individual  differences  and  behavioristic 
approaches,  became  more  pronounced. 

Starting  in  the  mid-1970s,  the  U.S.  military 
services  funded  several  learning-skills  research 
projects  that  were  to  occupy  a  prominent 
place  in  the  scientific  literature.  For  example. 
Dansereau,  Actkinson,  Long,  and  McDonald 
(1974),  under  contract  with  the  Air  Force, 
reviewed  the  research  literature  related  to  aca¬ 
demic  learning.  From  that  basis,  Dansereau 
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and  his  colleagues  (e.g.,  Dansereau  et  al., 
1979)  developed  their  Learning  Strategy  Sys¬ 
tem,  which  they  partitioned  into  two  parts — 
Primary  Strategies  and  Support  Strategies. 
The  Primary  Strategies  consist  of  Compre¬ 
hension-Retention  and  Retrieval-Utilization, 
each  of  which  contains  the  following  compo¬ 
nents:  understanding,  recall,  digesting/detail¬ 
ing,  expansion,  and  review.  Adding  the  letter 
"M"  from  "mood"  to  the  first  letter  of  these 
Primary  Strategies  provides  the  well-known 
acronym,  MURDER.  The  Support  Strategies 
consist  of  goal  setting,  concentration  manage¬ 
ment,  monitoring  and  diagnosing,  mood  set¬ 
ting,  and  mood  maintenance. 

Shortly  thereafter,  in  the  late  1970s  and  early 
1980s,  Weinstein  and  her  co-workers  (Wein¬ 
stein  et  al.,  1980;  Weinstein,  Washington, 
Wicker,  Duty,  &  Underwood,  1980)  developed 
a  set  of  cognitive  learning  strategies  for  the 
Armv.  Like  Dansereau,  Weinstein  developed 
a  comprehensive  model  of  the  teaching¬ 
learning  process,  only  one  part  of  which 
involves  study  skills.  From  a  study  skills  per¬ 
spective,  Weinstein  and  Mayer  (1986)  defined 
eight  learning  strategies  identified  as  basic 
rehearsal,  c<  jlex  rehearsal,  basic  elabora¬ 
tion,  complex  elaboration,  basic  organization¬ 
al,  complex  organizational,  comprehension 
monitoring,  and  affective  and  motivational 
strategies  Currently  the  Army  has  incorpo¬ 
rated  five  learning  strategies  into  its  Job  Skills 
Education  Program:  time  management,  read¬ 
ing  strategies,  problem  solving  (mathematics), 
test  taking,  and  motivational  skills  (Hoffman, 
Hahn,  Hoffman,  &  Dean,  1988;  Wilson,  1990). 
The  work  of  Brown  and  her  co-workers  (e.g., 
Brown,  Bransfbrd,  Ferrara,  &  Campione,  1983) 
on  metacognition  also  has  stimulated  both 
research  and  application  in  this  area. 

The  Navy  took  a  different  tactic.  In  a  series 
of  studies  in  Navy  schools,  researchers  dif¬ 
ferentiated  two  types  of  reading,  called  Read¬ 
ing  to  Do  and  Reading  to  Learn.  Students 
were  asked  to  report  which  activities  "had 
helped  them  to  learn  the  information  in  a 
reading-to-learn  task"  (Sticht,  1979,  p.  279). 
Four  major  categories  were  identified:  reread- 
rehearse,  problem  solve-question,  relate- 
associate,  and  focus  attention.  In  his  review 
of  cognitive  psychology  research  in  the  mili¬ 
tary,  Wittrock  (1979.  p.  309)  commented  that 


"the  armed  services  are  in  the  forefront  of 
application  of  pnnciples  of  cognitive  psychol¬ 
ogy  to  instruction." 

Although  the  military  services  have  been 
productive  in  this  area,  there  has  been  no  defi¬ 
ciency  of  academic  research  activity  on  the 
topic  of  study  skills.  This  research  has  resulted 
in  the  development  of  several  instruments  to 
assess  study  skills.  Among  these  are  the  Moti¬ 
vated  Strategies  for  Learning  Questionnaire 
(MSLQ)  (Pintrich  et  al.,  1988);  the  Study  Atti¬ 
tudes  and  Methods  Survey  (SAMS)  (Bache¬ 
lor  &  Michael,  1988;  Nadson,  Michael,  & 
Michael,  1988);  the  Survey  of  Study  Habits  and 
Attitudes  (SSHA)  (Davou  &  McKelvie,  1984; 
Gadzella,  1982);  the  Study  Skills  Question¬ 
naire  (SSQ)  (Bartlett  &  Knoblock,  1988);  and 
the  Study  Process  Questionnaire  (SPQ)  (O'Neil 
&  Child,  1984).  Derry  and  Murphy’s  (1986) 
review  of  learning  methods  training  confirmed 
that  the  study  skills  topic  and  research  are 
expanding. 

Nevertheless,  several  major  considerations 
confront  the  topic  of  study  skills  assessment 
and  evaluation.  One  of  these  is  the  absence 
of  a  consensus  concerning  the  domain  of 
study  skills.  In  addition,  many  of  the  current 
diagnostic  instruments  address  issues  that  are 
not  germane  in  all  settings.  For  example, 
many  such  instruments  assume  that  a  student 
is  taking  general  education  courses  instead  of 
one  course,  as  is  common  in  military’  or  tech¬ 
nical  schools.  Others  assume  more  flexibility 
in  study  time  than  most  military  students 
have.  Finally,  some  study  skills  surveys — like 
the  SSQ,  because  of  its  specialized  vocabulary — 
were  designed  primarily  for  administration  by 
specialists.  For  these  reasons,  a  new  instru¬ 
ment  to  assess  student  skills  related  to  the 
study  process  was  devised  and  evaluated  in 
military  contexts.  Further.  Wade  and  Trathen 
(1989)  reported  that  most  of  the  early  research 
on  study  methods  failed  to  assess  the  relation¬ 
ship  between  study  techniques  and  recall. 
Lastly,  Weinstein  and  Underwood  (1985) 
reported  that  many  research  studies  have  used 
grade-point  average  (GPA)  as  a  criterion 
Whereas  GPA  is  an  unambiguous  criterion, 
traditionally  it  embodies  several  deficits, 
namely,  course  self-selection,  a  wide  variety 
of  course  requirements,  and  diverse  grading 
systems. 
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These  issues  were  addressed  in  the  research 
described  in  this  articJe.  More  generally,  this 
report  describes  the  development  of  a  study 
skills  survey  and  four  related  research  studies — 
two  addressing  its  reliability  and  two  validity 
studies. 


INSTRUMENT  DEVELOPMENT 

During  an  iterative  process,  NPRDC  research¬ 
ers  devised  a  model  of  study  activity  and 
related  factors  based  on  military  research 
reports  and  their  own  experiences  in  military 
school  settings.  The  objective  was  to  identify 
and  define  all  possible  categories  of  student 
behaviors  or  factors  that  relate  to  study  pen 
formance.  Once  the  initial  set  of  categories 
was  developed,  independent  lists  of  possible 
questions  were  written  and  categorized. 
Group  meetings  served  to  simplify  and  clarify 
both  the  dimensions  and  questions  for  use  in 
a  self-report  questionnaire.  During  the  pro¬ 
cess,  the  number  of  dimensions  was  ex¬ 
panded  and  the  questions  were  modified. 
Notable  goals  during  this  process  were:  (a) 
item  development  and  revision  consistent 
with  the  study  factor  definitions,  to  address 
the  continuously  revised  model,  (b)  preserv¬ 
ing  conceptual  distinctions  between  the  di¬ 
mensions,  (c)  maintaining  a  low  requirement 
for  readability,  so  that  the  eventual  survey 
could  be  used  widely,  (d)  keeping  questions 
as  short  as  possible,  in  consideration  of  the 
respondent,  and  (e)  use  of  a  single  and  sim¬ 
ple  response  format. 

Study  factors,  not  just  study  skills,  served 
as  the  focus  because  important  aspects  of  an 
individual's  study  behavior  may  not  be  clas¬ 
sified  as  a  skill.  A  good  example  is  test  anxi¬ 
ety.  Spielberger  (e.g.,  Spielberger,  Gonzalez, 
&  Fletcher,  1979)  has  been  particularly  active 
in  promoting  the  concept  of  test  anxiety  as 
an  essential  component  of  learning  strategies. 
Additionally,  although  some  researchers  may 
not  consider  motivation  a  study  skill,  both 
Dansereau  (1985)  and  McCombs  and  Dob- 
rovolnv  (1982)  have  effectively  used  motiva¬ 
tional  training  in  their  systems.  Similarly, 
McKeachie  and  his  co- workers  (e.g.,  Pintrich 
et  al.,  1988)  emphasize  a  motivational  com¬ 
ponent  because  their  Motivated  Strategies  for 


Learning  Questionnaire  has  55  motivational 
items  and  55  cognitive  items.  Given  that  fac¬ 
tors  other  than  study  skills  have  been  impor¬ 
tant  for  study,  this  instrument  was  identified 
as  the  NPRDC  (Navy  Personnel  Research  and 
Development  Center)  Study  Factors  Survey. 

Table  1  lists  the  16  conceptually  indepen¬ 
dent  study  factors  and  their  operational  defi¬ 
nitions.  Whereas  all  16  study  factors  are 
conceptually  distinct,  some  dimensions  were 
expected  to  be  related  empirically,  perhaps  in 
different  populations,  for  two  reasons.  First, 
one  dimension  often  may  imply  another 
dimension.  For  example,  memorization  may 
be  much  easier  and  thus  more  useful  for  stu¬ 
dents  who  do  not  have  test  anxiety.  Second 
and  more  importantly,  students  with  good 
academic  backgrounds  may  develop  and  use 
several  of  these  study  factors  automatically, 
whereas  students  with  poor  academic  back¬ 
grounds  might  use  few  or  none  of  them. 
Although  the  development  of  distinct  factors 
was  a  goal,  the  conceptual  differentiation  of 
characteristics  among  the  study  factors  was 
viewed  as  the  more  critical  objective. 

The  unadministered  version  of  the  NPRDC 
Study  Factors  Survey  consisted  of  16  study  fac¬ 
tors  as  assessed  by  98  items  using  a  4-point 
response  scale  that  ranged  from  Almost  Always 
(4)  to  Almost  Never  (1).  This  number  of  factors, 
16,  is  larger  than  typically  reported  by  other 
study  skills  researchers;  however,  the  level  of 
specificity  in  each  researcher's  model  usually 
accounts  for  such  differences.  The  survey  was 
evaluated  for  both  reliability  and  validity.  Each 
of  the  two  reliability  estimations  and  two 
methods  of  validity  assessment  are  described 
individually  below. 


INITIAL  RELIABILITY  ASSESSMENT 

Method 

Subjects 

Because  the  scales  had  no  previous  empirical 
analysis,  reliability  was  assessed  using  a  two- 
step  process.  The  survey  was  initially  admin¬ 
istered  at  two  Navy  schools  (N  =  1 17  and  127) 
that  train  new  enlistees  in  electrical/electronic 
occupations  following  their  entry-level  train- 
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TABLE  1  □  Study  Factor  Definitions 


Anxiety 

Competition 


Concentration 

Elaboration 

Graphic  Study  Aids 
Group  Study 
Mastery  Beliefs 
Memorization 


Generalized  fear  associated  with  learning  or  testing. 

A  tendency  to  compare  one's  performance  to  that  of  classmates.  The 
perception  of  one's  performance  as  being  evaluated  in  companson 
toothers. 

The  ability  to  focus  on  learning  despite  either  internal  or  external 
distractions. 

The  use  of  any  of  several  methods  to  integrate  and  understand 
information. 

The  appreciation  and  use  of  charts,  figures,  and  tables  for  study. 

The  tendency  to  interact  with  other  students  during  study. 

Attitudes  relating  academic  achievement  to  personal  effort. 

The  use  of  any  of  several  mental  devices  to  store  and  later  recall 
information. 


Motivation 

Organization 

Questioning 

Review 

Self-Monitoring 

Study  Resource  Management 
Test  Anticipation 
Test  Strategy 


The  level  of  drive  or  perceived  incentive  to  either:  (a)  complete  training 
at  this  school  or  (b)  leam  assigned  training  materials. 

The  use  of  structure  and  preparation  of  materials  to  be  learned. 

The  verbal  process  of  seeking  clarification. 

A  selective  examination  of  previously  read  course  materials  to  enhance 
learning. 

A  mental  review  of  cognitive  process  to  ensure  it  is  logical  and 
compatible  with  previous  knowledge  and  assumptions. 

The  foresight  to  maximize  environmental  resources  to  permit  study. 

The  ability  to  predetermine  test  content. 

The  use  of  techniques  during  a  test  to  maximize  the  number  of  correct 
responses. 


TABLE  2  □  Two  Reliability  Estimates  of  the  NPRDC  Study  Factors  Survey 


TWO  SCHOOLS  ALL  FIVE  SCHOOLS 


Scale 

Items 

Alpha 

Alpha 

Mean 

S.D. 

Anxietv 

5 

.662 

.712 

11.66 

2.94 

Competition 

3 

.657 

.626 

8.74 

2.23 

Concentration 

7 

.649 

.669 

19.51 

3.30 

Elaboration 

6 

.734 

.731 

15.40 

3.41 

Graphic  Study  Aids 

3 

.681 

.680 

9.10 

1.79 

Group  Study 

5 

.910 

.903 

11.06 

4.01 

Masterv  Beliefs 

7 

.609 

.585 

16.66 

2.35 

Memorization 

5 

.600 

.632 

14.65 

2.60 

Motivation 

8 

.738 

.732 

27.10 

3.41 

Organization 

8 

.748 

.728 

22.39 

4.14 

Questioning 

3 

.729 

.727 

9.86 

1.94 

Review 

5 

.718 

.650 

15.88 

2.50 

Self-Monitonng 

8 

.706 

.668 

22.69 

3.51 

Study  Resource  Management 

5 

.727 

.716 

14.68 

2.81 

Test  Anticipation 

5 

.676 

.645 

14.07 

2.67 

Test  Strategy 

3 

.594 

.625 

8.95 

2.16 

N  range 

237-241 

688-700 
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ing  in  boot  camp.  The  typical  student  in  these 
two  schools  is  just  out  of  high  school,  has  a 
high  school  diploma,  and  may  be  considered 
an  average  or  slightly  below  average  student. 

Results 

Based  on  the  initial  administration  of  the 
Study  Factors  Survey,  items  were  deleted  from 
the  scales  until  no  further  improvement  in  reli¬ 
ability  estimates  was  obtained.  No  item  was 
added  to  any  scale.  After  item  deletion,  the 
values  of  coefficient  alpha  for  the  16  scales 
ranged  from  .594  to  .910  and  the  number  of 
items  per  scale  ranged  from  3  to  8.  These  sta¬ 
tistics  are  shown  in  the  second  and  third  col¬ 
umns  of  Table  2. 


SECOND  RELIABILITY  ASSESSMENT 
Method 
Subjects 

As  part  of  a  larger  Navy-wide  project,  NPRDC 
Study  Factor  Survey  forms  were  administered 
at  and  received  from  three  additional  Navy 
schools  <iV  =  62,  98,  and  301,  reflecting  their 
relative  enrollment  rates).  These  schools  also 
train  enlisted  personnel  in  a  variety  of  tech¬ 
nical  occupations  immediately  following  boot 
camp  training. 

Results  and  Discussion 

Table  2  also  shows  the  coefficient  alpha  esti¬ 
mates  of  internal  consistency  that  were  ob¬ 
tained  using  the  data  from  all  five  Navy 
schools  and  the  scale  means  and  standard 
deviations  obtained  for  these  schools.  As 
shown,  the  coefficients  fluctuated  somewhat 
between  the  first  and  second  reliability  assess¬ 
ments;  however,  most  remained  relatively 
stable.  For  the  entire  sample,  coefficient  alpha 
ranged  from  .585  to  .903,  whereas  most  values 
were  in  the  moderate  .6  and  .7  ranges.  The 
form's  total  internal  consistency  estimate  was 


.918.  Adding  items  that  had  been  deleted  in 
the  previous  reliability  analysis  did  not  in¬ 
crease  the  more  recent  alpha  estimates. 

Whereas  a  few  of  the  scales  demonstrated 
relatively  low  reliability  estimates,  most  of 
them  are  considered  acceptable  or  very  good. 
The  a  priori  dimensions  are  logically  compat¬ 
ible  with  the  results  from  the  internal  consis¬ 
tency  analysis.  Moreover,  these  reliability 
estimates  are  consistent  both  with  the  previ¬ 
ous  analysis  and  with  the  reliability  estimates 
for  commercial  study  skill  instruments  as 
reported  in  Weinstein  and  Underwood  (1985). 

FIRST  VALIDITY  STUD/ 

Method 

Generally,  students  in  Navy  enlisted  schools 
attend  classes  daily  and  are  tested  at  least  once 
a  week.  There  are  typically  from  20  to  25 
multiple-choice  items  per  test.  The  tests  are 
knowledge  based  and  performance  is  mea¬ 
sured  using  percentage  correct.  Test  perfor¬ 
mance  standards  are  fairly  rigid,  typically  80% 
correct.  Those  students  who  do  not  meet  test 
performance  standards  usually  are  either  put 
on  academic  probation,  set  back  to  another 
class,  or  dismissed  from  the  school.  For  this 
study,  test  scores  from  the  four  most  difficult 
tests  in  one  of  the  five  Navy  schools  (N  =  117) 
served  as  an  index  of  student  performance. 
More  specifically,  the  mean  percentage  of  cor¬ 
rect  answers  on  at  least  three  of  the  four  most 
difficult  tests  was  used  as  an  index  of  student 
performance. 

The  major  reasons  for  using  this  perfor¬ 
mance  index  were:  (a)  scores  on  several  knowl¬ 
edge  tests  are  more  representative  of  academic 
performance  than  any  single  test  score;  (b) 
although  the  means  and  standard  deviations 
differed  slightly,  the  difficult  nature  of  all  the 
tests,  the  consistent  course  theme,  and  the  use 
of  percentage  scoring  allowed  a  reasonable 
combination  of  the  scores;  (c)  one  score  could 
be  missing  (e.g.,  for  a  student  who  was  dis¬ 
missed  or  set  back  to  another  class)  without 
discarding  that  poor  performer's  data;  and  (d) 
unlike  traditional  GPA  scores,  these  test  scores 
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derived  from  homogeneous  testing  and  per¬ 
formance  standards.  Two  other  performance 
measures  were  available.  These  dichotomous 
indices  (0  =  no;  1  =  yes)  indicated  whether 
a  student  was  assigned  to  academic  probation 
or  set  back  to  another  class,  either  event  sig¬ 
naling  poor  academic  performance. 


Results  and  Discussion 

Correlations  were  computed  between  the  16 
study  factors  and  the  three  performance  cri¬ 
teria,  the  results  of  which  are  provided  in  Table 
3.  The  statistics  in  Table  3  show  that  all  but 
one  of  the  significant  correlations  were  in  the 
expected  direction.  The  exception  was  Test 
Strategy.  Five  other  study  factors  were  related 
significantly  to  test  performance:  Competition, 
Concentration,  Memorization,  Motivation,  and 
Questioning.  Thus,  those  students  who  were 
more  competitive  or  motivated,  had  higher 
concentration  or  memorization  skills,  asked 
more  questions,  or  were  low  on  test  strategy 
skills  performed  significantly  better  on  the 
most  difficult  tests  in  this  school.  The  relatively 


large  correlations  (greater  than  .4)  for  the  Com¬ 
petition  and  Concentration  scales  should  be 
noted. 

To  explore  the  contribution  of  the  signifi¬ 
cant  study  factors  to  test  performance,  a 
regression  analysis  was  performed.  The  results 
are  shown  in  Table  3.  A  subsequent  stepwise 
regression  analysis  included  three  of  the  six 
significant  factors  and  produced  an  R{ 3,  88)  = 
.618.  Thus,  the  joint  contribution  of  Compe¬ 
tition,  Concentration,  and  Test  Strategy  sig¬ 
nificantly  and  independently  accounted  for 
38.2%  of  the  test  performance  variance. 

Further,  students  who  were  placed  on  aca¬ 
demic  probation  (21.4%)  reported  low  scores 
for  two  of  the  study  factors:  Competition  (r 
=  -  .346,  p  <  .001)  and  Memorization  (r  = 
-  .206,  p  =  .031).  Also,  two  of  the  scales — 
Competition  and  Memorization — were  signif¬ 
icantly  correlated  with  both  test  performance 
and  academic  probation,  thus  providing  some 
indication  of  the  relative  importance  of  these 
study  factors  to  school  performance.  Students 
who  were  set  back  for  academic  reasons 
(15.4%)  were  distinguished  only  by  their  poor 
Motivation  (r  =  -  .217,  p  ~  .023)* 


TABLE  3  □  Study  Factors  and  Test  Performance  Analyses 


Stale 

FOUR  TESTS 

REGRESSION  ANALYSIS 

r 

p<r) 

Beta 

t-test 

p<t> 

Anxiety 

-.000 

Competition 

.443 

.000 

.398 

4.26 

.000 

Concentration 

.420 

.000 

.262 

2.81 

.006 

Elaboration 

.071 

Graphic  Study  Aids 

.055 

Group  Study 

-.147 

Mastery  Beliefs 

.094 

Memonzation 

.362 

.000 

.146 

1.44 

153 

Motivation 

.196 

.048 

-.054 

-.511 

.611 

Organization 

.093 

Questioning 

.214 

.028 

.047 

.464 

.644 

Review 

.135 

Self-Momtonng 

.056 

Study  Resource  Management 

.075 

Test  Anticipation 

.072 

Test  Strategv 

-.213 

.031 

-.284 

-3.231 

.002 

Sole  r  »  correlation  coefficient,  p\r)  ~  probability  of  r  under  the  null  hypothesis:  recorded  only  if  *  .05  Beta  is  the 
standardized  regression  coefficient  tor  the  significant  vanables  model  which  is  evaluated  by  the  t-test  and  its  associated 
probability 
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SECOND  VALIDITY  STUDY 

Method 

A  second  validity  analysis  examined  the  rela¬ 
tionships  between  the  study  factors  and  test 
performance  while  controlling  for  the  effect 
of  student  ability.  This  issue  is  germane 
because  one  could  argue  that  most  or  all  of  a 
student's  test  performance  could  be  accounted 
for  by  ability,  and  that  some  or  most  of  the 
study  factors  simply  represent  alternate  mea¬ 
sures  of  ability. 

A  good  index  of  ability  to  perform  well  in 
Navy  schools  is  provided  by  students'  compos¬ 
ite  scores  on  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB).  Every  enlisted  per¬ 
son  who  enters  the  U.S.  military  takes  the 
ASVAB,  and  the  battery  of  10  tests  has  been 
shown  repeatedly  to  serve  as  a  reliable  measure 
of  students'  academic  ability.  The  current  ver¬ 
sion  of  the  ASVAB  assesses  knowledge  and 
skills  in  10  areas:  Word  Knowledge,  Arithme¬ 
tic  Reasoning,  Coding  Speed,  Numerical 
Operations,  Math  Knowledge,  Mechanical 
Comprehension,  General  Science,  Electron¬ 
ics  Information.  Auto-Shop  Information,  and 
Paragraph  Comprehension.  Moreover,  each 
school  has  a  minimum  ASVAB  composite  cri¬ 
terion  score  for  entry  into  the  school.  Further 
details  about  the  ASVAB  can  be  found  in 
Wegner  and  Ree  (1986). 

Thus,  one  answer  to  the  question  of  whether 
abilitv  serves  to  moderate  the  effect  of  study 
factors  on  test  performance  is  to  describe  the 
proportion  of  variance  in  test  performance  that 
is  not  associated  with  the  ASVAB  composite 
but  is  associated  with  the  study  factors.  In 
other  words,  what  is  the  correlation  between 
each  study  factor  and  test  performance  after 
controlling  for  (or  partihoning)  the  effects  of 
student  ability? 

Subjects 

The  sample  of  301  students  who  completed 
the  Studv  Factors  Survey  for  the  reliability 
analvsis  was  used  for  this  analysis.  Perfor¬ 
mance  data  obtained  from  the  school  con¬ 
sisted  of  test  scores  with  the  highest  failure 


rates — three  early  in  the  course,  three  approx¬ 
imately  halfway  through  the  course,  and  three 
near  the  end  of  the  course.  Not  all  subjects 
had  taken  all  of  the  tests  at  the  time  the  data 
were  received.  Scores  were  available  for  241 
students  who  completed  the  three  most  dif¬ 
ficult  tests  given  early  in  the  course,  159  who 
also  completed  the  tests  halfway  through  the 
course,  and  80  of  them  completed  the  three 
tests  near  the  course  end.  ASVAB  composite 
scores  were  available  for  most  of  the  students. 


Results 

Table  4  reports  the  results  of  the  test  perfor¬ 
mance  analysis  for  the  early  tests.  Columns 
two  and  three  of  Table  4  identify  the  signifi¬ 
cant  correlation  coefficients  between  the  study 
factors  and  the  test  performance  criterion. 
Seven  of  the  16  study  factors  were  significant. 

The  column  headed  "Partial  r"  in  Table  4 
reports  the  significant  partial  correlation  coef¬ 
ficients  between  each  study  factor  and  test  per¬ 
formance,  controlling  for  ability  (the  ASVAB 
composite  score).  Student  ASVAB  composite 
scores  ranged  from  191  to  268  and  had  a  mean 
value  of  233.03  and  a  standard  deviation  of 
14.65.  Four  of  the  relationships  remained 
significant  (p  *  .05;  one-tailed)  after  the 
variance  attributed  to  ability  was  partitioned. 
These  findings  indicate  that,  irrespective  of 
their  academic  ability,  those  students  who  had 
low  anxiety  levels  as  well  as  relatively  good 
concentration  or  memorization  skills  per¬ 
formed  significantly  better  on  the  three  most 
difficult  tests  early  in  the  school.  In  addition, 
students  who  indicated  that  they  made  rela¬ 
tively  little  use  of  test  strategies  also  per¬ 
formed  better. 

To  explore  the  contribution  of  the  signifi¬ 
cant  study  factors  to  test  performance,  a 
regression  analysis  was  performed.  The  results 
are  shown  in  the  last  three  columns  of  Table 
4.  A  subsequent  stepwise  regression  analy¬ 
sis  included  three  of  the  factors  and  produced 
an  R(3,238)  =  .379.  Thus,  the  joint  contribu¬ 
tion  of  Anxiety,  Memorization,  and  Test  Strat¬ 
egy  significantly  and  independently  accounted 
for  14.4%  of  the  test  performance  variance. 
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TABLE  4  □  Study  Factors  and  EartyTest  Performance  Analyses 


Scale 

EARLY  TESTS 

PARTIAL  r 

REGRESSION  ANALYSIS 

r 

p(r) 

r 

p(r) 

Beta 

t-test 

p<t) 

Anxietv 

-.321 

.000 

-.230 

.001 

-.266 

-.415 

.000 

Competition 

.146 

.024 

Concentrahon 

.231 

.000 

.175 

.010 

.081 

1.22 

.224 

Elaboration 

.103 

Graphic  Study  Aids 

.105 

Group  Study 

-.134 

.040 

Masterv  Beliefs 

.116 

Memorization 

.172 

.007 

.162 

.014 

.102 

.161 

.108 

Motivation 

.067 

Organizabon 

-.047 

Quesboning 

.093 

Review 

.063 

Self-Monitonng 

.136 

.035 

Study  Resource  Management 

-.117 

Test  Anhcipahon 

.046 

Test  Strategy 

-.199 

.002 

-.166 

.012 

-.122 

-1.97 

.049 

Note  r  «  correlation  coefficient;  p(r )  «  probability  of  the  r  under  the  null  hypothesis.  Partial  correlation  controlling  for  the 
effect  of  ability;  recorded  only  if  «  .05.  Beta  is  the  standardized  regression  coefficient  for  the  significant  variables  model 
which  is  evaluated  by  the  t-test  and  its  associated  probability. 


Performance  data  for  the  three  difficult  tests 
midway  through  the  course  indicated  that 
three  of  the  study  factors  were  significant. 
These  were  Anxiety  (r  =  -  .215,  p  -  .007), 
Concentration  (r  =  .214,  p  =  .007),  and 
Study  Resource  Management  (r  =  -  .204,  p 
=  .010).  Correlations  for  the  later  performance 
measures  indicated  that  four  study  factors 
were  significant;  Anxiety  (r  =  -.301,  p  - 
.007),  Concentration  (r  =  .261,  p  =  .020), 
Mastery'  Beliefs  (r  *  .259,  p  =  .022),  and 
Questioning  (r  =  .319,  p.  =  .005).  Partial  corre¬ 
lations  were  not  performed  for  the  later  perfor¬ 
mance  measures  because  of  insufficient  data. 

DISCUSSION 

The  results  from  two  validity  analyses  indi¬ 
cate  that: 

1.  Study  factors  correlated  significantly 
with  test  performance  in  two  independent 
schools. 

2.  Different  study  factors  correlated  signifi¬ 
cantly  at  each  school. 

3.  Although  more  study  factors  were  signifi¬ 
cant  at  the  beginning  of  the  course,  differ¬ 
ent  study  factors  correlated  significantly  at 
different  portions  of  the  course. 


4.  The  study  factor  relationship  to  test  per¬ 
formance  was  significant  irrespective  of  stu¬ 
dent  ability. 

5.  Regression  analyses  supported  the  inde¬ 
pendent  contribution  of  several  study  fac¬ 
tors  to  test  performance  at  two  schools. 

When  these  vaiidity  studies  are  viewed  to¬ 
gether,  a  pattern  emerges.  Of  the  16  study  fac¬ 
tors,  11  were  significant  in  one  or  both  schools. 
What  is  most  conspicuous  in  this  pattern  are 
the  findings  for  Memorization,  Concentration, 
and  Questioning.  Memorization  was  related 
to  test  performance  in  both  schools,  related 
to  academic  probation,  and  survived  the  par¬ 
tial  correlation  analysis.  Likewise,  Concentra- 
bon  was  related  to  test  performance  in  both 
schools  and  also  survived  the  partial  conela- 
bon  analysis.  Because  Quesboning  was  sig¬ 
nificantly  related  to  test  performance  only  late 
in  the  course,  it  did  not  undergo  the  partial 
correiahon  analysis,  but  was  significant  in 
both  schools.  These  study  factors  deserve 
attenbon,  at  least  for  technical  training  schools. 
Moreover,  these  study  factors  may  have  impli- 
cabons  for  the  educabon-technology  interface 
in  that  technology  may  be  able  to  affect  how 
students  memorize,  concentrate,  and  quesbon 
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Th .  findings  for  Anxiety  and  Motivation 
may  have  been  school  specific.  Anxiety  was 
related  to  all  three  performance  measurements 
in  one  school,  and  it  also  survived  the  partial 
correlation  analysis.  In  the  other  school.  Moti¬ 
vation  was  related  to  both  test  performance 
and  academic  probation.  It  is  not  dear  whether 
Anxiety  and  Motivation  are  antithetical  or  sim¬ 
ply  school  spedfic.  Other  study  factors  either 
did  not  pass  the  partial  correlation  test  (Com¬ 
petition,  Group  Study,  Self-Monitoring)  or 
were  not  subjected  to  it  (Mastery  Beliefs  and 
Study  Resource  Management).  Also,  although 
Competition  failed  the  partial  correlation  test, 
its  affects  should  be  investigated  further  due 
to  the  influence  of  this  factor  in  military  train¬ 
ing.  This  topic  deserves  further  study. 

Reasoning  suggests  that  factors  such  as 
Anxiety,  Concentration,  Memorization,  and 
Questioning  have  important  ties  to  the  study- 
leaming-performance  process.  These  analyses 
provide  solid  evidence  for  the  consideration 
of  such  factors  in  learning  environments.  Only 
three  study  factors  provided  a  significant 
result  in  an  unexpected  direction — Group 
Study  (which  did  not  pass  the  partial  corre¬ 
lation  analysis).  Study  Resource  Management, 
and  Test  Strategy.  The  item  content  for  the  Test 
Strategy  scale  concerns  the  order  of  answer¬ 
ing  easy  versus  difficult  questions  on  a  test. 
It  may  be  that  in  these  and  similar  schools  it 
is  necessary  or  functional  to  answer  test  ques¬ 
tions  from  first  to  last  rather  than  answer  the 
easy  questions  first.  This  topic  also  deserves 
further  study. 

The  three  key  issues  associated  with  study 
skills  research  are  their  selection  and  develop¬ 
ment,  training,  and  effectiveness  assessment 
(Dansereau,  1985).  This  article  addressed  two 
of  those  issues  in  a  military  context.  Based  on 
these  findings,  evaluations  should  be  under¬ 
taken  to  determine  whether  study  factor  train¬ 
ing  can  be  implemented  to  influence  test 
performance.  For  example,  training  may  be 
indicated  for  students  whose  scores  fall  below 
selected  cutoff  points  for  Concentration,  Mem¬ 
orization,  Questioning,  or  other  skills.  Dob- 
rovolny,  McCombs,  Sc  Judd  (1980),  Hoffman 
et  al.  (1988),  and  McCombs  Sc  Dobrovolny 
(1982)  are  resources  that  describe  military 
training  aids  for  several  study  factors,  includ¬ 


ing  Reading  Comprehension,  Concentration, 
Memorization,  and  Motivation.  A  fourth  key 
issue  that  should  be  raised  and  addressed  by 
future  research  is  whether  study  skills/factors 
are  compensatory  or  necessary.  Two  regres¬ 
sion  analyses  provided  evidence  that  s  ime  of 
these  study  factors  made  independent  con¬ 
tributions  to  test  performance.  Yet,  if  students 
are  low  on  one  or  a  few  factors,  can  other  fac¬ 
tors  compensate  for  that  deficit?  Such  issues 
require  dedicated  research  effort  if  we  are  to 
make  advances  into  understanding  the  study- 
leam-performance  process  and  how  technol¬ 
ogy  can  contribute  to  that  process. 

Although  the  NPRDC  Study  Factors  Survey 
was  designed  to  assess  study  factors  in  mili¬ 
tary  technical  schools,  the  factors  obtained  in 
this  research  have  high  face  validity,  seem  to 
be  generic,  and  thus  may  underlie  learning 
and  study  in  the  general  population.  Conse¬ 
quently,  the  findings  from  this  research  effort 
may  have  applications  to  non-military  aca¬ 
demic  settings.  Moreover,  research  is  required 
to  delineate  whether  different  learning  objec¬ 
tives  or  different  topics  of  varying  content  and 
difficulty  require  different  study  factors  for 
effective  learning.  □ 
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